Adapting RANSAC SVM to Detect Outliers for Robust Classification

In this paper we address the problem of classifying objects where some of the labels in the training data are noisy. This is a common scenario and can be caused by the difficulty of annotation or inadvertently due to human error. In this paper, we consider the wrongly annotated examples to be outliers and try to formulate a robust outlier identification algorithm. The task of learning a model in the presence of noise has been traditionally solved by the RANSAC algorithm[1]. RANSAC has also been adapted as RANSAC SVM by Nishida and Kurita [3]. The RANSACSVM method selects random subsets of the training data and trains small SVMs on them, using the rest of the training data as validation sets. It then chooses the SVM with the smallest validation error to approximate the full training set. However, if the training data is noisy, the validation sets are corrupted, and a faulty submodel may be chosen as optimal. To address this problem, we propose a modification to RANSAC SVM thereby achieving robustness to noise. The detailed algorithm is given in Algorithm 1. We are initially given a training set S of n examples from which we draw small random subsets of size k. For each such subset, we train a SVM to obtain a weight vector wi using the standard support vector formulation given by: